Out in the World: What Did The Robot Hear And See?

It is now well established that jointattention is a key capability for sociallyinteracting robots (Brooks et al., 1999,Kaplan and Hafner, 2004, Scassellati, 1999,Itti, 2003). It is also a key component forepigenetic robotic applications in general.This subject has been widely discussedand we present here one specific technicalimprovement for joint attention which relieson image segmentation.In the new Talking Robots (Baillie, 2004)experiment that we have started, followinga successful reimplementation of the Sony’sTalking Heads (Steels, 1998) experiment onAibo ERS7, we try to have two robots inter-acting to evolve a shared repertoire of syn-chronized behaviors, or “games”.This leads to a dynamic version of the Talk-ing Heads, where the interaction protocol,or language game, is not predefined in theagents. As usually with experiments involvingsymbol grounding and social interaction, andfor these experiments in particular, it is es-sential that the two robots can establish jointattention and share a common representa-tion of their surrounding environment, whilethey are looking at the same scene. Manytechniques can be used to achieve this stableshared representation. We have chosen to fo-cus on segmentation-based algorithms, whichprovide interesting shapes together with vec-tors of features that can be easily extractedand proved useful in the context of our exper-iment.However, when using segmentation algo-rithms, a slight change in the viewpoint, oreven the residual camera noise, is enough tosignificantly change the result of the imagepartition, possibly leading to completely dif-ferent perceptions of the same scene.We have developed an original measure toassess the stability of a segmentation algo-rithm and we have used it on a set of algo-rithms to automatically determine the moststable partition for a given scene. This ap-proach is different from classical methods usedto estimate the quality of a segmentation al-gorithm, where the result of the algorithmis compared to an ideal perfect segmentationdone by hand. In our approach, the measureis done automatically, involves only stabilityconsiderations and could lead to interestingimprovements whenever joint attention usingsegmentation is required.We quickly present in the poster the back-ground of the Talking Robots experiment andwhy joint attention and image segmentationstability is an important issue for us. We in-troduce then two stability measures and showsome results on natural scenes from our ex-periments in the lab and test scenes used tocontrol image parameters. The influence ofseveral image characterizations (noise, num-ber of objects, luminosity,...) is carefully re-viewed. An example (fig.1) is given below,showing the influence of noise on typical im-ages.Using the fact that a given algorithm can beranked according to its stability score, whichis calculated online assuming that the sceneitself is static, a general method of algorithmswitching is introduced and used in the ex-periment with different kind of algorithms:region growing, recursive histogram splitting,CSC (Priese and Rehrmann, 1993) and split& merge (CIS). We show how this methodsignificantly improves the convergence speedin the experiment and conclude on the gen-erality of our approach to facilitate certainaspects of joint attention, when it relies onsegmentation.